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METHOD FOR GENERATING A FINAL SIGNAL FROM A NEAR-END 
SIGNAL AND A FAR-END SIGNAL 

Inventor: Brent W. Edwards and Sunil Puria 

1. FIELD OF THE INVENTION 

[0001] The present invention relates generally to the field of telephone sets 

connected to a telephone network and more specifically to the problem of using a 
telephone in a noisy environment. 

2. BACKGROUND 

[0002] When a person uses a telephone in a noisy environment such as a noisy 

room, an airport, a car, a street comer or a restaurant, it can often be difficult to hear the 
person speaking at the other end the "far-end") of the connection over the 
background noise present at theJistener's location (Le., the "near-end"). Due to the 
variability of human speech, the far-end speaker's voice is sometimes intelligible over the 
near-end background noise and sometimes unintelligible. Moreover, the noise level at 
the near-end may itself vary over time, making the far-end speaker's voice level adequate 
sometimes and at inadequate at other times, 

[0003] Although some telephones provide for control of the volume level of the 

telephone loudspeaker (/.e., the earpiece), such control is often imavailable. Moreover, 
manual adjustment of a volume control by the listener is undesirable since, as the 
backgroxmd noise level changes, the user will be required to re-adjust the manual volume 
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control in an attempt to maintain a preferred listening level. Generally, it is more 
desirable to provide an automatic (/.e., adaptive) control mechanism, rather than requiring 
the listener to adjust a manual volume control. One solution, which attempts to address 
this problem, has been proposed in U.S. Patent No. 4,829,565, issued on May 9, 1989 to 
Goldberg. Goldberg discloses a telephone with an automated volume control whose gain 
is a function of the level of the background noise. The use of either conventional manual 
volume controls or an automatic mechanism such as that disclosed in Goldberg fails to 
adequately solve the background noise problem. In particular, these approaches fail to 
recognize the fact that by amplifying the signal that supplies the handset receiver (/.e., the 
loudspeaker), the side tone is also amplified. (The side tone is a well-known feed-through 
effect in a telephone. A portion of the input signal from the handset transmitter, Le., the 
microphone, is mixed with the far-end signal received from the network. The resultant, 
combined signal is then supplied to the handset loudspeaker.) Since the side tone 
contains background noise, the background noise is disadvantageously amplified with the 
far-end signal. By amplifying both the far-end signal and the noise together, the 
degrading effect of the noise can actually become worse. 

[0004] Another solution that attempts to address this problem has been proposed 

20 in U.S. Pat. No. 5,526,419, issued on June 11, 1996 to Allen. Allen proposes a telephone 
that separates the far-end signal into a plurality of subbands and amplifies each subband 
by a gain factor. The gain factor that is applied to individual subbands of the far-end 
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signal is proposed to be a function of a received signal indicative of the background 
noise. Allen then proposes combining the sidetone with the amplified far-end signal. 
[0005] While Allen enhances the quality of the far-end signal, Allen does not 

enhance the near-end signal. Thus, a listener will find it difficult to hear his own voice 
5 because of the background noise. 

[0006] Thus, a method is needed that enhances the quality of both the far-end 

signal and the near-end signal. 



:3 3 . SUMMARY OF INVENTION 

10 [0007] One embodiment of the invention is a method of processing a far-end 

ft 

=is signal and a near-end signal to produce a final signal, the far-end signal containing 

n 

'n speech, the near-end signal containing speech and background noise. The method 

"3 includes: determining an amplification gain based upon the near-end signal; removing a 

portion of the backgroimd noise from the near-end signal to create a noise-reduced near- 

;? R 

15 end signal; combining the far-end signal with the noise-reduced near-end signal to create 
a combined signal; and amplifying the combined signal by the amplification gain to 
create the final signal. 

[0008] Another embodiment of the invention is a method of processing a far-end 

signal and a near-end signal to produce a final signal, the far-end signal containing 
20 speech, the near-end signal containing speech and background noise. This method 

includes: separating the near-end signal into a first near-end subband signal and a second 
near-end subband signal; determining a first amplification gain based upon the first near- 
end subband signal; determining a second amplification gain based upon the second near- 
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end subband signal; removing a portion of the background noise from the near-end signal 
to create a noise-reduced near-end signal; combining the far-end signal with the noise- 
reduced near-end signal to create a combined signal; separating the combined signal into 
a first combined subband signal and a second combined subband signal; amplifying the 
5 first combined subband signal by the first amplification gain to create a first amplified 
subband signal; amplifying the second combined subband signal by the second 
amplification gain to create a second amplified subband signal; and combining the first 
combined subband signal and the second combined subband signal to create the final 
i!3 signal. 

■=f| 1 0 [0009] Another embodiment of the invention is yet another method of processing 

u1 

I a far-end signal and a near-end signal to produce a final signal, the far-end signal 

i y 

I ri containing speech, the near-end signal containing speech and background noise. This 

0 method includes: separating the near-end signal into a first near-end subband signal and a 

second near-end subband signal; determining the masking level of noise of the first near- 

Ul 

15 end subband signal; determining the masking level of noise of the second near-end 

subband signal; estimating the masking level of noise of a third near-end subband signal 
based upon the masking level of noise of the first near-end subband signal and the 
masking level of noise of the second near-end subband signal; determining a first 
amplification gain based upon the masking level of noise of the first near-end subband 

20 signal; determining a second amplification gain based upon the masking level of noise of 
the second near-end subband signal; determining a third amplification gain based upon 
the masking level of noise of the third near-end subband signal; removing a portion of the 
backgroimd noise from the near-end signal to create a noise-reduced near-end signal; 
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combining the far-end signal with the noise-reduced near-end signal to create a combined 
signal; separating the combined signal into a first combined subband signal, a second 
combined subband signal, and a third combined subband signal; amplifying the first 
combined subband signal by the first amplification gain to create a first amplified 
subband signal; amplifying the second combined subband signal by the first amplification 
gain to create a second amplified subband signal; amplifying the third combined subband 
signal by the first amplification gain to create a third amplified subband signal; and 
combining the first combined subband signal, the second combined subband signal, and 
the third combined subband signal to create the final signal. 

[0010] Still other embodiments of the invention are program storage devices 

containing computer readable instructions that when executed by a digital signal 
processor perform any of the above methods. 

[0011] Other embodiments of the invention include telephones that include such 

program storage devices. 

[0012] Still another embodiment of the invention is a communication device. The 

communication device includes: a transmitter/receiver adapted for a communication 
medium; control circuitry coupled to the. transmitter/receiver that controls transmission, 
reception and control of audio signals; a speaker coupled to the control circuitry that 
renders audio signals audible; and a microphone coupled to the control circuitry that 
transforms soimds into a sidetone signal. The control circuitry includes: a noise filter that 
receives the sidetone signal and produces a noise reduced sidetone signal; and an 
amplifier that combines an audio signal received from the transmitter/receiver with the 
noise reduced sidetone signal to produce a combined signal, amplifies the combined 
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signal according to a function responsive to the background noise in the sidetone, and 
provides an enhanced audio signal to the speaker. 



4. BRIEF DESCRIPTION OF THE FIGURES 

[0013] Figure 1 presents a flow diagram of one embodiment of the invention. 

[0014] Figure 2 presents one embodiment of a digital signal processor based 

system. 

[0015] Figure 3 presents a block diagram of program modules. 

[0016] Figure 4 presents another embodiment of the invention. 

[0017] Figure 5 presents still another embodiment of the invention. 

[0018] Figure 6 presents a flow diagram of another embodiment of the invention. 

[0019] Figure 7 presents a flow diagram of still another embodiment of the 

invention. 

5. DESCRIPTION OF THE PREFERRED EMBODIMENTS 

[0020] The following description is presented to enable any person skilled in the 

art to make and use the invention, and is provided in the context of a particular 
application and its requirements. Various modifications to the disclosed embodiments 
will be readily apparent to those skilled in the art, and the general principles defined 
herein may be applied to other embodiments and applications without departing fi-om the 
spirit and scope of the present invention. Thus, the present invention is not intended to be 
limited to the embodiments shown, but is to be accorded the widest scope consistent with 
the principles and features disclosed herein. 
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[0021] Embodiments of the present invention improve the signal-to-noise ratio of 

a far-end signal in the near-end listener's ear when the near-end listener is using a 
telephone in a noisy environment. In addition, embodiments of the present invention 
improve the signal-to-noise ratio of the near-end signal in the near-end listener's ear 
when the near-end listener is using the telephone in the noisy environment. 
[0022] One embodiment of the invention, as shown in Figure 1 , is a method of 

processing a far-end signal and a near-end signal. The far-end signal typically contains a 
signal that may have been communicated over a telephone network to a (near-end) 
listener. The far-end signal may have been communicated over a telephone network such 
as the POTS (plain old telephone service) network or a more modem network such as 
ISDN (integrated services digital network) or FDDI (fiber distributed data interface). 
Alternatively, the far-end signal may have been communicated over a wireless network 
such as the cellular telephone network. The near-end signal, which in some embodiments 
of the invention would be the previously discussed sidetone, typically contains the 
listener's voice and often contains background noise. 

5.1 Separating the Near-end Signal into Subbands 

[0023] Referring to block 101 of Figure 1, the near-end signal is first separated 

into a plurality of near-end subband signals. For example, the near-end signal may be 
separated into a 500 Hz ± 25 Hz subband signal, a 1 KHz + 50 Hz subband signal, and a 
3 KHz + 150 Hz subband signal. In some embodiments of the invention, such as 
discussed immediately above, the subband widths are equal to 10% of the subband center 
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frequency. In other embodiments of the invention, the widths of the subbands may be 
selected to equalize the equivalent rectangular bandwidth (ERB) of each subband signal. 
[0024] The separation of the near-end signal into a plurality of near-end subband 

signals may be performed by passing the near-end signal through a plurality of analog 
filters, such as band pass filters. Alternatively, the near-end signal may be passed 
through a plurality of digital filters, such as FIR and/or IIR filters. In still other 
embodiments of the invention, the separation of the near-end signal may be passed 
through a Fast Fourier Transfer (FFT) procedure running on a digital signal processor. 

5.2 Determining the Masking Level of Noise in the Near-end Subband Signals 
[0025] Referring to block 102 of Figure 1, after the near-end signal has been 
separated into a plurality of near-end subband signals, the masking level of noise in each 
subband can be determined. For example, the masking level of noise in a particular near- 
end subband signal can be determined by calculating the sound pressure level (dB) of that , 
near-end subband signal. 

5.3 Estimating the Masking Level of Noise of Additional Subband Signals 

[0026] Referring to block 103 of Figure 1, after the masking level of noise in the 

above near-end subband signals has been determined, the masking level of noise in one 
or more near-end subband signals may be estimated. For example, the masking level of 
noise of a near-end subband signal can be estimated by interpolating between the 
masking levels of noise of two other near-end subband signals. Alternatively, the 
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masking level of noise of the near-end subband signal can be estimated by extrapolating 
the masking levels of noise of two other near-end subband signals. 
[0027] In some embodiments of the invention, the masking level of noise of one 

or more near- end subband signals can be combined with a noise model to estimate the 
masking levels of noise of additional near-end subband signals. For example, the 
masking level of noise of a single near-end subband signal, when combined with a noise 
model of a typical automobile, can be used to estimate the masking level of additional 
near-end subband signals in an automobile. 

[0028] The masking levels of noise of a plurality of near-in subband signals can 

also be utilized to select from one or more noise models. For example, the masking 
levels of noise of a plurality of near-end subband signals, can be compared to the 
masking levels of noise of a plurality of corresponding subband signals, /.e., same 
subband center frequency and same subband frequency width, in various noise models, to 
determine which noise model is most similar to the near-end noise. After the noise 
model has been selected, then the masking level of noise of one or more near-end 
subband signals can be combined with the selected noise model to estimate the masking 
levels of noise of additional near-end subband signals. 

[0029] The above methods may be utilized to accurately estimate the masking 

levels of noise of approximately 20 near-end subband signals based upon three or four 
measured near-end subband signals. 

5.4 Determining Subband Amplification Gains 
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[0030] Referring to blocks 104 and 105 of Figure 1, after the masking levels of 

noise of the plurality of near-end subband signals has been determined and/or estimated, 
subband amplification gains can then be determined. In one embodiment of the 
invention, the masking levels of noise of near-end subband signals (dB) are converted to 
sound pressure levels above the threshold of hearing audibility (dBHL) using equations 
known by those of skill in the art. Then, subband amplification gains are determined by 
using various protocols, such as but not limited to, the Fig. 6 protocol, the National 
Acoustics Laboratories' NAL-NLl protocol, the Independent Hearing Aid Fitting 
Forum's protocol, the Desired Sensation Level input/output (DSL [i/o]) protocol, or the 
Cambridge protocol. 

5.5 Removing a Portion of the Background Noise fi-om the Near-end Signal 
[0031] Referring to block 106 of Figure 1, a portion of the backgroimd noise is 

removed fi*om the near-end signal to create a noise-reduced near-end signal. Many 
methods are known by those of skill in the art for removing a portion of such background 
noise. For example, a portion of the background noise fi-om the near-end signal can be 
removed by filtering the near-end signal with a high-pass filter. Alternatively, a portion 
of the background noise can be removed by filtering the near-end signal with a high-pass 
filter and suppressing the DC component of the near-end signal. 
[0032] Still other embodiments of the invention use the well know spectral 

subtraction technique to remove a portion of the background noise from the near-end 
signal. See for example, Boll, "Suppression of Acoustic Noise in Speech using Spectral 
Subtraction," IEEE Trans, on Acoustics, Speech and Signal Processing, Vol. ASSP-27, 
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No. 2, April 1979, p. 113. Generally, the spectral subtraction technique estimates the 
spectral content of "clean" speech by explicitly subtracting the spectral content of 
background noise from speech signals that include background noise. One 
implementation of the spectral subtraction technique is proposed in U.S. Patent No. 
5,742,927 issued on April 21, 1998 to Crozier, which is hereby incorporated by reference. 
[0033] Still other embodiments of the invention utilize a technique known as 

spectral scaling to remove a portion of the background noise from the near-end signal. 
See for example, Eger, "A Nonlinear Processing Technique for Speech Enhancement," 
Proc. ICASSP 1983 (IEEE) pp. 18A.1.1-18.A.1.4 and U.S. Patent No. 5,133,013 issued 
on July 21, 1992 to Munday. 

[0034] Still other embodiments of the invention utilize other known noise 

suppression techniques, such as cepstral subtraction and Weiner filtering to remove a 
portion of the background noise from the near-end signal. 

5.6 Combining the Far-end Signal with the Noise-reduced Near-end Signal 
[0035] Referring to block 107 of Figure 1, the noise-reduced near-end signal is 
combined with the far-end signal to create a combined signal. In one embodiment of the 
invention, the two signals are combined within a digital signal processor. In another 
embodiment of the invention, the two signals are combined in an adder of conventional 
design. 

5.7 Amplifying the Combined Signal 
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[0036] Referring to blocks 108-1 10 of Figure 1, the combined signal is then 

processed by a multiband amplifier that has been set to amplify different subbands of the 
combined signal by the subband amplification gains determined in Section 5.4. 
Multiband amplifiers are well known by those of skill in the art. See, for example, U.S. 
5 Patent No. 6,198,830, issued on March 6, 2001 to Holbe and U.S. Patent No. 5,526,419, 
issued on Jxine 11, 1996 to Allen. 

[0037] In one illustrative multiband amplifier, referring to block 108 of Figure 1, 

the combined signal will first be separated into a plurality of combined subband signals. 
i^f For example, in one embodiment the combined signal may be separated into 20 combined 

1"'^ 10 subband signals. Then, referring to block 109, each of these signals is amplified by a 
subband amplification gain to create amplified combined subband signals. Finally, 

i y 

I referring to block 110, the amplified combined subband signals are combined to create a 

□ final signal. 

m 
m 

pi 15 5.8 Outputting the Final Signal 

1=4 

[0038] Referring to block 111 of Figure 1, after a multiband amplifier has 

processed the combined signal, the resulting final signal is output through a speaker, such 
as a telephone handset speaker. 

20 5.9 Digital Signal Processor Implementation 

[0039] Figure 2 presents one embodiment of a digital signal processor based 

system for performing the methods described above. The apparatus includes a 
microphone 201 for converting a user's voice and backgroimd noise into a near-end 
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signal. The output of the microphone 201 is coupled to a conventional preamp 202 that is 
also coupled to a first analog-to-digital converter 203. The analog-to-digital converter 
203 is conventional. The output of the first analog-to-digital converter 203 is coupled to 
a conventional multiplexer 205. The output of the multiplexer 205 is coupled to a digital 
signal processor 206 that is programmed to perform one of the methods described above. 
The output of the digital signal processor 206 is coupled a digital-to-analog converter 
207. The digital-to-analog converter 207 is conventional. The output of the digital-to 
analog converter 207 is coupled to a conventional speaker 208. A second analog-to- 
digital converter 204 receives the far-end signal. The output of the second analog-to- 
digital converter 204 is coupled to the multiplexer 205. 

[0040] Figure 3 presents a block diagram of program modules that could be 

included in a digital signal processor 206 that was programmed to perform some of the 
embodiments of the invention. The demultiplexer 301 receives the output firom 
multiplexer 205 and separates the near-end signal 302 fi-om the far-end signal 303. 
Subband separator module 304 receives the near-end signal and, as discussed in Section 
5.1, generates near-end subband signals. The masking level of noise determiner modules 
305 and 306 receive the near-end subband signals, which, as discussed in Section 5.2, 
determine the masking level of noise in each near-end subband signal. In some 
embodiments of the invention, the outputs of the masking level of noise determiner 
modules 305 and 306 are provided to the masking level of noise estimator module 307 
(data path between masking level of noise determiner modules 305 and 306 and the 
masking level of noise estimator module 307 is not shown). Thus, as described in 
Section 5.3, an additional masking level of noise can be estimated. Amplification gain 
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determiner modules 308, 309, and 310 receive the outputs of the masking level of noise 
determiner modules 305 and 306 and the masking level of noise estimator module 307. 
As discussed in Section 5.4, the amplification gain determiner modules 308, 309, and 310 
provide amplification gains to the multiband amplifier 313. 

[0041] The near-end signal 302 is also provided to the noise reducer module 311. 

As discussed in Section 5.5, the noise reducer module removes a portion of the 
background noise fi"om the near-end signal and creates a noise-reduced near-end signal 
314. This signal 314 is received by an adder module 312, which also receives the far-end 
signal 303 fi"om the demultiplexer 301 . As discussed in Section 5.6, the adder module 
combines the far-end signal with the noise-reduced near-end signal to create a combined 
signal 315. The multiband amplifier 313 receives the combined signal 315. As discussed 
in Section 5.7, the multiband amplifier 313 then multiplies subbands of the combined 
signal 315 to generate a final signal 316. 

5.10 Other Embodiments of the Invention 

[0042] In other embodiments of the invention, a user would preprogram subband 

amplification gains into a telephone. In one embodiment of the invention, the subband 
amplification gains could be programmed via the telephone keypad. In another 
embodiment of the invention, the subband amplification gains could be encoded in the 
far-end signal. In still other embodiments of the invention, the subband amplification 
gains could be input into the telephone via voice recognition. The subband amplification 
gains may be based upon the user's hearing ability and/or the anticipated background 
noise that is present when the telephone is typically used. 
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[0043] In some embodiments of the invention, the amplification gain detemiiner 

modules 308, 309, and 310 first determine subband amplification gains as discussed in 
Section 5.4. Next, each module retrieves a preprogrammed subband amplification gain. 
Then, the module provides the multiband amplifier 3 1 3 with the larger of either the 
5 determined subband amplification gain or the preprogrammed subband amplification 
gain. 

[0044] Figure 4 presents a block diagram of another embodiment of the invention. 

In this embodiment, the microphone converts sovmd that includes background noise into a 
Q near-end signal. The near-end signal is provided to a noise reduction system and a 

0$ 1 0 speech enhancement system. The noise reduction system eliminates a portion of the 
;:fl background noise fi-om the near-in signal and produces a noise-reduced near-end signal. 

An adder combines the far-end signal and the noise-reduced near-end signal to produce a 

combined signal. In some embodiments of the invention, the speech enhancement system 
1=4 amplifies the combined signal. In other embodiments of the invention, the speech 

Q 1 5 enhancement system separates the combined signal into a plurality of combined subband 

signals. In such embodiments, the combined subband signals are amplified using one of 

the multiband compression methods discussed above, and then combined into a final 

signal, which is provided to the speaker. 

[0045] Still another embodiment is presented in Figure 5. Figure 5 presents a 

20 block diagram of a communication device, such as a cellular telephone. The 

communication device includes a transmitter/receiver and control circuitry that is coupled 
to the transmitter/receiver. The control circuitry controls transmission, reception, and 
processing of audio signals. The communication device also includes a speaker and a 



RXSD 1008-1 



Page 15 



microphone that are coupled to the control circuitry. The speaker renders audio signals 
audible and the microphone converts sound into a sidetone signal. The control circuitry 
includes a noise filter that receives a sidetone signal from a microphone and produces a 
noise-reduced sidetone. The control circuitry also includes an amplifier that combines an 
audio signal received firom the transmitter/receiver with the noise-reduced sidetone to 
produce a combined signal. The amplifier also amplifies the combined signal according 
to a function that is responsive to the backgroimd noise in the sidetone signal. Further, 
the amplifier provides an enhanced audio signal to the speaker. 

5.11 Conclusion 

[0046] The foregoing descriptions of embodiments of the present invention have 

been presented for purposes of illustration and description only. They are not intended to 
be exhaustive or to limit the present invention to the forms disclosed. For example, the 
methods shown in Figures 6 and 7 present additional embodiments of the invention. 
Similarly, a digital signal processor that contains computer readable instructions that 
when executed by the digital signal processor perform any of the above methods is 
encompassed by the invention. Additionally, the invention is not intended to be limited 
to the specifically disclosed methods for determining amplification gains based upon 
soxmd pressure levels or sound pressure levels above the threshold of hearing audibility. 
The disclosed methods are only illustrative. Other methods known by those skilled in the 
art for determining amplification gains may be utilized. Further, the invention is not 
intended to be liniited to specifically disclosed methods for removing a portion of the 
background noise fi"om the near-end signal. The invention is likewise not intended to 
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require a digital signal processor. Any device, such as a micro-controller or a 
microprocessor, that is capable of receiving digital data and outputting digital data may 
be utilized to perform the above methods. The disclosed methods are only illustrative 
and other methods known by those skilled in the art for removing background noise may 
be utilized. 

[0047] Accordingly, many modifications and variations will be apparent to 

practitioners skilled in the art. Additionally, the above disclosure is not intended to limit 
the present invention. The scope of the present invention is defined by the appended 
claims. 



RXSD 1008-1 



Page 17 



